Integrating Inductive Neural Network Learning and Explanation-Based Learning

Many researchers have noted the importance of combining inductive and analytical learning, yet we still lack combined learning methods that are effective in practice. We present here a learning method that combines explanation-based learning from a previously learned approximate domain theory, together with inductive learning from observations. This method, called explanation-based neural network learning (EBNN), is based on a neural network representation of domain knowledge. Explanations are constructed by chaining together inferences from multiple neural networks. In contrast with symbolic approaches to explanation-based learning which extract weakest preconditions from the explanation , EBNN extracts the derivatives of the target concept with respect to the training example features. These derivatives summarize the dependencies within the explanation, and are used to bias the inductive learning of the target concept. Experimental results on a simulated robot control task show that EBNN requires significantly fewer training examples than standard inductive learning. Furthermore , the method is shown to be robust to errors in the domain theory, operating effectively over a broad spectrum from very strong to very weak domain theories. 1 The Problem Analytical learning methods such as explanation-based learning (EBL) [DeJong and Mooney, 1986], [Mitchell et a/., 1986] use prior knowledge to explain and then generalize from observed training data. While such methods may dramatically reduce the number of training examples needed for successful generalization, in their pure Figure 1: Combining inductive and analytical learning: In the ideal case, a learning system deals with all levels of domain theories, i.e., it is robust with respect to severe errors therein. It operates purely inductively if no domain theory is available or the domain theory is random, and purely analytically if the domain theory is perfect. form they require correct and complete prior knowledge of the domain. In contrast, inductive learning methods require no such prior knowledge, but rely instead on many more training examples to guide generalization , together with some syntactic inductive bias. One of the major open problems in machine learning is to combine analytical and inductive learning in order to gain the benefits of both approaches: reduced requirement for training data, and robustness with respect to poor prior knowledge. Figure 1 illustrates the spectrum of domain theories over which a general learning system should be able to operate. At present, we have inductive learning methods that operate well at the leftmost point on the spectrum , in which no domain theory …